Recent technological and scientific advances have generated wide interest in the possibility of creating a brain–machine interface (BMI), particularly as a means to aid paralyzed humans in communication. Advances have been made in detecting neural signals and translating them into command signals that can control devices. We now have systems that use externally derived neural signals as a command source, and faster and potentially more flexible systems that directly use intracortical recording are being tested. Studies in behaving monkeys show that neural output from the motor cortex can be used to control computer cursors almost as effectively as a natural hand would carry out the task. Additional research findings explore the possibility of using computers to return behaviorally useful feedback information to the cortex. Although significant scientific and technological challenges remain, progress in creating useful human BMIs is accelerating.


 
 


The Brain Machine Interfaces Program


 represents a major DSO thrust area that will comprise a multidisciplinary, multipronged approach with far reaching impact. The program will create new technologies for augmenting human performance through the ability to noninvasively access codes in the brain in real time and integrate them into peripheral device or system operations. Focus will be on the following areas: 

  1. Extraction of neural and force dynamic codes related to patterns of motor or sensory activity required for executing simple to complex motor or sensory activity (e.g., reaching, grasping, manipulating, running, walking, kicking, digging, hearing, seeing, tactile). This will require the exploitation of new interfaces and algorithms for providing useful nonlinear transformation, pattern extraction techniques, and the ability to test these in appropriate models or systems.

2. Determination of necessary force and sensory feedback (positional, postural, visual, acoustic, or other) from a peripheral device or interface that will provide critical inputs required for closed loop control of a working device (robotic appendage or other peripheral control device or system).

3. New methods, processes, and instrumentation for accessing neural codes noninvasively at appropriate spatiotemporal resolution to provide closed loop control of a peripheral device. This could include both fundamental interactions of neural cells, tissue, and brain with energy profiles that could provide noninvasive access to codes (magnetics, light, or other). 

4. New materials and device design and fabrication methods that embody compliance and elastic principles, and that capture force dynamics that integrate with neural control commands. These include the use of dynamic materials and designs into working prototypes. 

5. Demonstrations of plasticity from the neural system and from an integrated working device or system that result in real time control under relevant conditions of force perturbation and cluttered sensory environments from which tasks must be performed (e.g., recognizing and picking up a target and manipulating it). 

6. Biomimetic implementation of controllers (with robotics or other devices and systems) that integrate neural sensory or motor control integrated with force dynamic and sensory feedback from a working device or system. The first phase of the program may include dynamic control of simple and complex motor or sensory activity directly using neural codes integrated into a machine, device, or system. Simple actions considered include using a robotic arm or leg to sense a target, reach for it and manipulate it, throw or kick an object at a target, or recognize a sensory input and responding to it (visual, acoustic) directly through input/output brain integration. More complex activity may include issues related to force or sensory perturbation in more complex environments. 
 
The Michigan BMI Project


The University of Michigan-based Project on Human Augmentation Through Advanced Brain-Machine Interfaces seeks to develop a scalable and robust “intelligent” system for coordinated control of a group of semi-autonomous robots through a direct brain interface. The project goals are centered on establishing proof-of-concept demonstrations of brain-machine interface systems to augment a person’s ability to achieve coordinated, task-driven control of a large set of objects within a very high degree-of-freedom and dynamic environment. 

The project involves a coordinated set of animal experiments that are coupled to the development of new technologies for establishing neural interfaces and signal processing for controlling robots. The project has three coordinated research thrusts: (i) systems technologies, (ii) non-invasive neural interface technologies, and (iii) testing and validation using sophisticated animal and robot experiments. The project is organized into eight major task groups: 1. System Development and Administration, 2. Neuro-imaging, 3. Peripheral Nerve Bus Extension, 4. Neural Interface Microsystems, 5. Rat experiments, 6. Monkey experiments, 7. Algorithms and Architectures, and 8. Robot Technologies. 

Brain-Machine Interfaces: Current and Future Applications
Summary: 
The purpose of Brain-Machine Interface (BMI) research is to create outputs from or inputs to the brain that are additional to those that occur naturally. There are two possible directions of data flow (data from the brain to outside machinery and data to the brain from outside machinery). In whole, this research suggests that the brain is a much more adaptable organ than it is commonly thought to be.


Brain to machinery data flow (Data out) 
The most immediate and practical goal of BMI research is to create a mechanical output from neuronal activity. Data out BMIs capitalize on the fact that motor cortex activity occurs approximately 300ms before its associated motor response. Therefore, a mechanical system that is to mimic a natural motor system has this period of time to create motor output while allowing the subject to maintain the experience of natural motor processing. One major goal of BMI research is to create a system that will allow patients who have damage between their motor cortex and muscular system to bypass the damaged route and activate outside mechanisms by using neuronal signals. This would potentially allow an otherwise paralyzed person to control a motorized wheelchair, computer pointer, or robotic arm by thought alone.


Apparatuses
The data-out BMI consists of multi-neuron population recordings in motor cortex, the output of which is electronically decoded. The neural implants typically consist of a four by four millimeter multi-electrode (25-100+ electrode) array that is implanted into the area of motor cortex that is most active during the motor task of interest (such as moving an arm). 
Data-out BMIs are a recent phenomenon because they involve several technological feats that are of recent origin: predicting (via FRMI) where the motor activity of interest occurs, the ability permanently implant single cell recording devices into the motor cortex, and the ability to make mathematical and mechanical sense of the neural output in the short time period necessary for such control. The first experiments into BMI were particularly limited by electrodes that had a short usable life (in the range of hours after being implanted) before they lost their ability to transmit useful signals. This was due to both buildup of organic materials on the wire ends and because the wires shapes tended to damage surrounding brain areas during animal movement. Newer, blunt-tipped, flexible, Teflon-coated stainless steel microwires produce very little damage and allow permanent implantation. Permanent implantation is necessary for complex testing because the interface has to be tuned specifically for each subject and for each implantation.
After neural outputs are obtained, additional machinery decodes the neural signals using algorithms (that are based upon the output associations that the system acquired during training) that derive the intended motor output of the system. Although decoding technology is currently customized for each application, the platforms have several characteristics in common. The hardware mechanisms allow for parallel, real-time processing based on the parallel data that was used to create the decoding algorithms. These mathematical algorithms derive a mechanical signal from the brain output and are adjusted by higher order algorithms to compensate for variations caused by the animal feedback learning. That is, the algorithms are sensitive to the fact that the subject's realizes that its thoughts are producing external cues, and the algorithms adjust in real time to compensate for this subject adjustment.


Studies that have demonstrated a functioning data-out BMI

Chapin, Moxon, Markowitz, and Nicolelis (1999) taught rats to obtain water via a bar-pressing task. The bar was electronically connected to a motor that controlled a robot arm. The robot arm had a resting position near the rat's head. When the bar was pressed, the arm rotated away from the cage and toward a water dropper. The degree of bar press was proportional to the speed of the arm movement. A small cup at the tip of the arm was able to catch water from the water dropper, and the rats learned how to coordinate the bar press with the arm movement to obtain water. The task was designed to get the rats to think about the action that the bar was having on the arm, not on the immediate task of pressing the bar: if the rats simply held the bar down then the cup would rotate past the water dropper, and if they never released the bar then the arm would never come back to its resting position near the cage so that the rat could drink the water.
Chapin et al. used the data from the activity of populations of neurons in the motor cortex (16 electrodes, 32 neurons) and thalamus (8 electrodes, 16 neurons) to create neuronal population functions that could explain a rat's forearm movement. This allowed for neural information to be processed by these algorithms in real time, and for control of the robot arm to be controlled by the output of these brain-derived signals. The input for the motor was then switched away from the bar and to the output of the neuronal population functions. Four of six of the rats (not coincidentally, those with the highest number of active task-related neurons) were routinely able to use the brain activity-derived signal to accomplish the task. The rats were theoretically able to control the arm with brain activity alone (e.g. ignoring the bar), and this dissociation between the bar movement and robot arm movement did occur, but only after extended practice with brain activity-derived robot arm control.
Wessberg, Stambaugh, Kralik, Beck, Chapin, Kim, Biggs, Srinivasan, and Nicolelis (2000) found that they could reliably predict and replicate the lever-press activity of an owl monkey in real time using only the signal derived from the monkey's brain. The monkey was trained to press a bar to the left in response to a light on the left and to the right to a light on the right. This study extended the findings of Chapin et al (1999) by adding two functional, robotic arms that moved simultaneously with the monkey's arm. In other words, the robotic arms acted like spatially separate third arms that were able to mimic the movement of the current arm. This study has obvious practical implications to the development of neural prosthetics, but it also allowed for the researchers to project the number of neurons that would need to be sampled to create accurate control of the prosthetic arm. They found that the number of neurons that needed to be sampled was roughly proportional to the accuracy of the arm-- such that 100 neurons will produce movements that are approximately 70% accurate and 500-700 neurons would have to be sampled to produce 95% accuracy. This suggests that neuroprotheses may be able to operate reliably by sampling a feasibly small number of neurons (in the hundreds), at least for the simple tasks with which they were tested.
Serruya, Hatsopoulos, Paninski, Fellows, & Donoghue (2002) demonstrated that signals from neural implants in the motor cortex of Rhesus monkeys could quickly and reliably be translated into a complex computer input to be used for a pointer interface. They also used a similar mechanical setup to Chapin et al (1999), but they trained Rhesus monkeys (that had been given motor implants) to play a pinball-like videogame using a handheld joystick. After input control of the videogame was switched away from the joystick and to the neurally-derived signal, the monkeys were not only able to continue to play the game, but eventually learned to play without a controller at all. Furthermore, the monkeys displayed evidence of learning how to use the neurally-derived signal more effectively. In other words, it appeared as if the monkeys not only accepted the new form of output, but adapted to it as well.
Using a far less mechanically complex system, Kennedy, Bakay, Moore, Adams, and Goldwaithe (2000) implanted a working data-out interface into the motor cortex of a completely paralyzed human patient. He implanted only two electrodes, but those electrodes were surrounded with neurotrophic factors that caused cortical tissue (which normally finds the electrodes disruptive or, at best, inert) to grow into the electrodes. After several weeks of neural growth, the patient was trained to "concentrate" to increase his neural signal, and through a relatively simple algorithmic system, he could reliably move a computer cursor horizontally across a computer screen. This study is important as a proof-of-concept for medical applications. It also demonstrates that humans are able to learn how to control their neural states even using new neural tissue-- a proof-of-concept that could make an almost unlimited variety of data-out applications feasible.


Speculative implications of data-out BMIs

The implications of data-out BMIs have been considered for decades (for a short review see Norman, 1993). Science fiction stories of a blurring line between human and machine are slowly being realized by studies such as those of Kennedy et al. Now that data-out BMIs are a limited reality, future applications will most likely be developed to attain higher bandwidth, higher accuracy, more reliability, and increased availability.
Perhaps the most interesting phenomenon will be the way that the human brain adapts to the new interface. Humans are well suited to the task of controlling and directing imagination, and this is the essence of producing neural output. With a high bandwidth interface and adequate training, there is a possibility that humans will come to view brain-derived output as an extension of their bodies. External cognition theorists and philosophers (sometimes referred to as "activity theorists") cite the fact that humans already strongly associate the use of certain tools with their bodies (Nardi, 1996; Heidegger, 1962) and some go as far as to say that using a neural-output arm would be similar to learning other cognitive tools (Norman, 1993). Although remotely controlled limbs are likely find their first uses with the disabled, it is easy to speculate that these neural outputs could guide other machinery and be used to enhance already-functioning human output systems. 


Machinery to brain data flow (Data in)

The research up to this point has concerned itself with the medically practical side of the BMI. Although creating an interface that starts with neural impulses and ends with mechanical actions is becoming more technologically feasible and the potential benefits to humanity are large, there may be uses for an interface that employs the opposite direction; one that that starts with a mechanical action and ends with neural impulses. Extra-sensory neural input has already been implemented in animals, although only for a very limited application. However, due to complicated ethical and practical problems, human implementation has not been seriously considered.


Apparatus
Data-in BMI implants typically consist of few electrodes. This is due to the fact that the electrodes are implanted for the purpose of electrical stimulation of neurons and a single current can have an effect on many neurons.
Demonstration of a working data-in BMI
Mechanical to neural BMI research has found at least one potentially useful application, that of remotely stimulated rat. Talwar, Xu, Hawley, Weiss, Moxon, & Chapin (2002) implanted three stimulating probes into the brains of rats. Two of the probes were implanted into the area of somatosensory cortex that receives input from the left and right whiskers. Stimulation of these probes is interpreted by the rat as a whisker touch. The third probe went into the medial forebrain bundle and was used as a way to reward the rat for following instructions. Probes were fed into a harness that contained the radio equipment and stimulating machinery that was necessary for the rat to receive remote input.
Rats were trained on a maze and were taught to turn left or right when stimulated by the appropriate left or right whisker probe. Rats were reinforced for following instructions with MFB stimulation. Rats were trained to interpret MFB stimulation alone (when it was not preceded by a whisker touch signal) as an indication to move forward. After training, the rats were able to generalize their lab training to tasks outside of the laboratory and were guided in real time via computer-controlled transmitters through a variety of difficult obstacles. It is suggested that these rats could be used for search-and-rescue tasks that require rat-like maneuverability that is not possible with modern robotics.
Data-in BMIs have never been implemented in humans. However, electrical brain-stimulating mechanisms have been implanted in humans for the treatment of epilepsy and to Parkinson's disease (Pearson, 2002). It would be inappropriate to label this a true data-in BMI because it does not input meaningful data. However these human applications serve as proof that simple stimulation from chronic implants is possible in the human brain and may be cited as evidence that other forms of human BMIs are feasible.


Speculation on Data-in BMIs

Although the data-in BMI is currently only a way of influencing the decisions of an animal based upon trained responses to broad neural stimulation, the far future of the BMI has captured the imagination of many scientific theorists. There is tremendous speculative interest in the possibility of chronically implanting a series of stimulating electrodes into a sensory area of human cortex. This implant could stimulate an existing system in a new way (such as adding infrared sensitivity to the visual system) or it could create a completely new form of human sensory experience by tying a portion of the brain to machinery that is attuned to a novel form of energy (such as a computer interface). If this novel system is eventually feasible, it will almost undoubtedly fall short of the hopes of most science fiction writers, because it would almost definitely require extensive training and perhaps implantation within a critical period of development (Norman, 1993).
Combining Data in and Data out systems for a natural feedback system
One major criticism of data-out neural implant systems that are designed to bypass the natural motor is that they do not allow for feedback of the movement to inform the person where his or her mechanically-controlled device may be and what it is experiencing. A true neuroprosthetic would take the current data-out system of deriving the mechanical intentions of neural signals and transmit this signal to a mechanical device that not only moves, but also contains tactile, visual, and proprioceptive sensors that can feed into the appropriate sensory brain areas. This system has been proposed as a distant goal, but has not yet been implemented.


Conclusion

The technology to create permanent BMIs is not even a decade old, and proof-of-concept tests have already demonstrated that with as few as two electrodes a brain can create a somewhat useful filtered signal, and, with many more electrodes, motion can be replicated with reasonable accuracy. Furthermore, there is evidence that the brain is not only able to produce this output, but is able to adapt to it as well. In the case of neural stimulation via electrodes, the brain responds to external input extraordinarily well, and this input can severely influence the behavior of the organism.